Engaging and disengaging for autonomous driving

ABSTRACT

Aspects of the present disclosure relate switching between autonomous and manual driving modes. In order to do so, the vehicle&#39;s computer may conduct a series of environmental, system, and driver checks to identify certain conditions. The computer may correct some of these conditions and also provide a driver with a checklist of tasks for completion. Once the tasks have been completed and the conditions are changed, the computer may allow the driver to switch from the manual to the autonomous driving mode. The computer may also make a determination, under certain conditions, that it would be detrimental to the driver&#39;s safety or comfort to make a switch from the autonomous driving mode to the manual driving mode.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/377,383, filed Apr. 8, 2019, which is a continuation of U.S.patent application Ser. No. 15/982,044, filed May 17, 2018, now issuedas U.S. Pat. No. 10,300,926, which is a continuation of U.S. patentapplication Ser. No. 15/785,623, filed Oct. 17, 2017, now issued as U.S.Pat. No. 10,000,216, which is a continuation of U.S. patent applicationSer. No. 15/498,886, filed Apr. 27, 2017, now issued as U.S. Pat. No.9,821,818, which is a continuation of U.S. patent application Ser. No.15/337,339, filed Oct. 28, 2016, now issued as U.S. Pat. No. 9,663,117,which is a continuation of U.S. patent application Ser. No. 15/075,878,filed Mar. 21, 2016, now issued as U.S. Pat. No. 9,511,779, which is acontinuation of U.S. patent application Ser. No. 14/746,268, filed Jun.22, 2015, now issued as U.S. Pat. No. 9,352,752, which is a continuationof U.S. patent application Ser. No. 14/333,839, filed Jul. 17, 2014, nowissued as U.S. Pat. No. 9,075,413, which is a continuation of U.S.patent application Ser. No. 14/095,226, filed Dec. 3, 2013, now issuedas U.S. Pat. No. 8,818,608, which is a continuation of U.S. patentapplication Ser. No. 13/792,774, filed on Mar. 11, 2013, now issued asU.S. Pat. No. 8,825,258, which claims the benefit of the filing date ofU.S. Provisional Patent Application No. 61/731,717 filed Nov. 30, 2012,the entire disclosures of which are hereby incorporated herein byreference.

BACKGROUND

Autonomous vehicles use various computing systems to aid in thetransport of passengers from one location to another. Some autonomousvehicles may require some initial input or continuous input from anoperator, such as a pilot, driver, or passenger. Other systems, forexample autopilot systems, may be used only when the system has beenengaged, which permits the operator to switch from a manual driving mode(where the operator exercises a high degree of control over the movementof the vehicle) to an autonomous driving mode (where the vehicleessentially drives itself) to modes that lie somewhere in between.

BRIEF SUMMARY

Aspects of the disclosure provide a method. In such aspects, the methodincludes receiving a request to switch a vehicle from a manual drivingmode to an autonomous driving mode. In response, protocol data isaccessed. This protocol data is used by a processor to assess the statusof the vehicle's environment, the vehicle and systems of the vehicle,and a driver and identify one or more conditions. The method alsoincludes accessing task data which associates conditions with tasks thatmay be performed by a driver to change the condition. The task data isused to generate a set of tasks. Each task of the set of tasks isdisplayed in an ordered sequence. Once the tasks are complete, theswitch from the manual driving mode to the autonomous driving mode mayproceed.

In one example, assessing the status of the vehicle's environment, thevehicle and systems of the vehicle, and a driver includes accessingsensor data, accessing detailed map information, communicating withvarious systems of the vehicle, and communicating with a remotecomputer.

The assessments may include, for example, one or more of determiningwhether the current or future weather would make autonomous drivingunsafe, uncomfortable for the vehicle's passengers or damage thevehicle, whether the vehicle's actual location agrees with the vehicle'slocation relative to the detailed map information, whether data receivedfrom the sensors of the detection system agrees with the correspondingdetailed map information, whether the vehicle is currently driving in anarea pre-approved for initiating the autonomous mode, whether thevehicle is currently driving in a lane pre-approved for initiating theautonomous mode, whether the roadway is paved (as opposed to dirt),whether the road is wide enough (as opposed to being too narrow for twovehicles to pass one another), whether the vehicle is on a straightroadway (as opposed to driving around a curve, down or up a hill, etc.),whether the vehicle is sufficiently centered in the lane, whether thevehicle is surrounded by or boxed in by other vehicles (for example, indense traffic conditions), whether the vehicle is currently in a schoolzone, whether the vehicle is facing oncoming traffic (for example,driving northbound in a southbound lane according to the detailed mapinformation), whether another vehicle is pulling into the vehicle's laneproximate to the vehicle, whether the vehicle is at least somepre-determined minimum distance from other vehicles or moving objects inthe roadway, whether the vehicle must make any small maneuvers to avoidnon-moving or slow moving objects in the roadway, whether the vehicle istraveling too fast or too slow, whether any upcoming maneuvers wouldprevent a switch from manual to autonomous driving mode, whether thevehicle can be driving in the autonomous mode for a minimum period oftime before a switch to manual mode is required, whether the variousfeatures of the autonomous driving system of vehicle are workingproperly, whether the vehicle has met minimum maintenance standards (forexample, oil changes are up to date), whether the vehicle's tires areproperly inflated, whether the vehicle's doors are closed, whether thevehicle is currently in “drive” (as opposed to “neutral,” “reverse,” or“park”), whether the vehicle is in two wheel drive (for example, on avehicle which allows for the driver to manually switch into four wheelor all wheel drive), whether the vehicle's automatic wipers arecurrently on, whether the vehicle's headlights or fog lights are on, andwhether no other warning indicators (such as check engine lights) arecurrently active, and whether the driver's seatbelt is properly buckled.

Other conditions that prevent a switch from manual to autonomous modemay include that the vehicle is not in “drive,” that the automaticwipers are not on, that the automatic lights are not on, and that thevehicle is not in a lane pre-approved for initiating autonomous driving.In such cases, the generated set of tasks may include putting thevehicle in drive, turning on automatic wipers, turning on automaticlights, and moving the vehicle to a lane pre-approved for initiatingautonomous driving. In this example, the sequence of displayed tasksincludes a first displayed task for putting the vehicle in drive, asecond displayed task for turning on automatic wipers, a third displayedtask turning on automatic lights, and a fourth displayed task moving thevehicle a lane pre-approved for initiating autonomous driving. The firstdisplayed task is displayed until confirmed completed. When the firstdisplayed task is completed, the second displayed task is displayeduntil the second task is completed. When the second displayed task iscompleted, the third displayed task is displayed until the third task iscompleted. When the third displayed task is completed, the fourthdisplayed task is displayed until the fourth task is completed.

In another example, the preventive conditions may include that thevehicle is not in “drive,” that the automatic wipers are not on, thatthe automatic lights are not on, that the vehicle is not in a lanepre-approved for initiating autonomous driving, that the vehicle is notcentered in a lane, that the vehicle is then moving too fast, and thatthe vehicle is too close to another object in the roadway. In thisexample, the generated set of tasks may include putting the vehicle indrive, turning on automatic wipers, turning on automatic lights, movingthe vehicle a lane pre-approved for initiating autonomous driving,centering the vehicle in the lane, slowing the vehicle down, andincreasing the distance between the vehicle and the another object inthe roadway.

In another example, the method also includes, before using the task datato generate the set of tasks, correcting by the processor any of theidentified one or more conditions pre-approved for correction by theprocessor. In another example, assessing the status is performedcontinuously while the tasks are being completed. In this regard,additional conditions may be identified and used to add additional tasksto the set of tasks.

In another example, the method may also include, after switching to theautonomous driving mode, maneuvering the vehicle in the autonomousdriving mode and receiving a request to switch from the autonomousdriving mode to the manual driving mode. This request may be notrecommended or denied when the vehicle is performing an action thatcannot be safely or easily transferred to the driver before the actionis complete. In this example and in response to an indication by theuser that the user wants to return to manual mode, a warning may bedisplayed to the driver when switching to manual driving mode is notrecommended.

Other aspects of the disclosure provide systems including a processorwhich performs some or all of the various features of the methodsdescribed above. Further aspects of the disclosure provide anon-transitory, tangible computer-readable storage medium on whichcomputer readable instructions of a program are stored. Theinstructions, when executed by a processor, cause the processor toperform some or all of the various features of the methods describedabove.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of a system in accordance with aspects ofthe disclosure.

FIG. 2 is an interior of an autonomous vehicle in accordance withaspects of the disclosure.

FIG. 3 is an exterior of an autonomous vehicle in accordance withaspects of the disclosure.

FIG. 4 is an illustration of a highway used by way of example inaccordance with aspects of the disclosure.

FIG. 5 is an example of map information in accordance with aspects ofthe disclosure.

FIG. 6 is another example of map information in accordance with aspectsof the disclosure.

FIG. 7A is a pictorial diagram of a system in accordance with aspects ofthe disclosure.

FIG. 7B is a functional diagram of a system in accordance with aspectsof the disclosure.

FIG. 8 is a series of screen images in accordance with aspects of thedisclosure.

FIG. 9 is a series of screen images in accordance with aspects of thedisclosure.

FIG. 10 is a flow diagram in accordance with aspects of the disclosure.

DETAILED DESCRIPTION

A vehicle may make a variety of determinations before allowing a driverto operate a vehicle in an autonomous driving mode, such as handing offcontrol of the vehicle to the vehicle's computer, or allowing the driverto switch from the autonomous driving mode to the manual driving mode.This may include conducting a series of environmental, system, anddriver checks and also providing a driver with a checklist of tasksbefore allowing the driver to hand off control of the vehicle to thevehicle's computer.

In one example, a computer associated with a vehicle having anautonomous driving mode and a manual driving mode may receive a requestto switch from a manual driving mode to an autonomous driving mode. Thecomputer may then respond by accessing protocol data and using it toassess status of the vehicle's current environment, the vehicle's likelyfuture environment, the vehicle, and the driver. The computer may thendetermine if these assessments have identified any conditions that thesystem uses to prevent vehicle from being placed into an autonomousdriving mode (hereafter, “preventive conditions”). If not, the computermay proceed with the switch from the manual to the autonomous drivingmode.

If, as a result of the assessments, the computer identifies one or morepreventive conditions, the computer may determine whether the computeror driver can change those identified conditions immediately or within acertain period of time. If the preventive conditions cannot be sochanged, the computer may display a failure message to the driver. Insome examples, this failure message may indicate the particular issuefor the failure.

If the computer can change the identified preventive conditions, thecomputer may take the actions necessary to do so. Even if the computeris otherwise capable of changing the preventive condition, the vehiclemay be designed to require the driver to change the condition. Forexample, the driver may be in a better position to assess if and how thepreventive condition can be changed. If no identified preventiveconditions remain, the computer may proceed with the switch from themanual to the autonomous driving mode.

If after the computer corrects any of the preventive conditions otherpreventive conditions remain, the computer may generate a correspondingtask or task for each of the remaining problem conditions. The computermay then display the tasks in order of priority to the driver. Forexample, the computer may be programmed to always display the tasks oneat a time in a predetermined order, or dynamically determine the orderin which the remaining tasks should be displayed. Once the task iscompleted, the computer may display the next task for completion. Thecompute may continue to display the tasks until each is corrected, andthe computer may then proceed with the switch from manual to autonomousdriving mode. In addition, this list may be recomputed periodically suchthat if one of the steps is completed while another step is being shown,then the completed task may be removed.

While the driver is performing the tasks, the computer may continue tomake the aforementioned assessments to identify any additionalpreventive conditions. These conditions may then cause the computer todisplay the error message, correct the additional preventive conditions,or generate and display additional tasks for completion by the driver.Again, if all of the preventive conditions have been corrected, thecomputer may proceed with the switch from the manual to the autonomousdriving mode.

The computer may similarly make assessments to determine whetherswitching the vehicle from the autonomous driving mode to the manualdriving mode is not currently recommended.

As shown in FIG. 1 , an autonomous driving system 100 in may include avehicle 101 with various components. While certain aspects of thedisclosure are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, busses, boats, airplanes,helicopters, lawnmowers, recreational vehicles, amusement park vehicles,farm equipment, construction equipment, trams, golf carts, trains, andtrolleys. The vehicle may have one or more computers, such as computer110 containing a processor 120, memory 130 and other componentstypically present in general purpose computers.

The memory 130 stores information accessible by processor 120, includinginstructions 132 and data 134 that may be executed or otherwise used bythe processor 120. The memory 130 may be of any type capable of storinginformation accessible by the processor, including a computer-readablemedium, or other medium that stores data that may be read with the aidof an electronic device, such as a hard-drive, memory card, ROM, RAM,DVD or other optical disks, as well as other write-capable and read-onlymemories. Systems and methods may include different combinations of theforegoing, whereby different portions of the instructions and data arestored on different types of media.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computer codeon the computer-readable medium. In that regard, the terms“instructions” and “programs” may be used interchangeably herein. Theinstructions may be stored in object code format for direct processingby the processor, or in any other computer language including scripts orcollections of independent source code modules that are interpreted ondemand or compiled in advance. Functions, methods and routines of theinstructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. For instance, although the claimedsubject matter is not limited by any particular data structure, the datamay be stored in computer registers, in a relational database as a tablehaving a plurality of different fields and records, XML documents orflat files. The data may also be formatted in any computer-readableformat. By further way of example only, image data may be stored asbitmaps comprised of grids of pixels that are stored in accordance withformats that are compressed or uncompressed, lossless (e.g., BMP) orlossy (e.g., JPEG), and bitmap or vector-based (e.g., SVG), as well ascomputer instructions for drawing graphics. The data may comprise anyinformation sufficient to identify the relevant information, such asnumbers, descriptive text, proprietary codes, references to data storedin other areas of the same memory or different memories (including othernetwork locations) or information that is used by a function tocalculate the relevant data.

The processor 120 may be any conventional processor, such ascommercially available CPUs. Alternatively, the processor may be adedicated device such as an ASIC or other hardware-based processor.Although FIG. 1 functionally illustrates the processor, memory, andother elements of computer 110 as being within the same block, it willbe understood by those of ordinary skill in the art that the processor,computer, or memory may actually comprise multiple processors,computers, or memories that may or may not be stored within the samephysical housing. For example, memory may be a hard drive or otherstorage media located in a housing different from that of computer 110.Accordingly, references to a processor or computer will be understood toinclude references to a collection of processors or computers ormemories that may or may not operate in parallel. Rather than using asingle processor to perform the steps described herein, some of thecomponents, such as steering components and deceleration components, mayeach have their own processor that only performs calculations related tothe component's specific function.

In various aspects described herein, the processor may be located remotefrom the vehicle and communicate with the vehicle wirelessly. In otheraspects, some of the processes described herein are executed on aprocessor disposed within the vehicle and others by a remote processor,including taking the steps necessary to execute a single maneuver.

Computer 110 may include all of the components normally used inconnection with a computer such as a central processing unit (CPU),memory (e.g., RAM and internal hard drives) storing data 134 andinstructions such as a web browser, an electronic display 152 (e.g., amonitor having a screen, a small LCD touch-screen or any otherelectrical device that is operable to display information), user input150 (e.g., a mouse, keyboard, touch screen and/or microphone), as wellas various sensors (e.g., a video camera) for gathering explicit (e.g.,a gesture) or implicit (e.g., “the person is asleep”) information aboutthe states and desires of a person.

In one example, computer 110 may be an autonomous driving computingsystem incorporated into vehicle 101. FIG. 2 depicts an exemplary designof the interior of an autonomous vehicle. The autonomous vehicle mayinclude all of the features of a non-autonomous vehicle, for example: asteering apparatus, such as steering wheel 210; a navigation displayapparatus, such as navigation display 215 (which may be a part ofelectronic display 152); and a gear selector apparatus, such as gearshifter 220. The vehicle may also have various user input devices 140 inaddition to the foregoing, such as touch screen 217 (which may be a partof electronic display 152), or button inputs 219, for activating ordeactivating one or more autonomous driving modes and for enabling adriver or passenger 290 to provide information, such as a navigationdestination, to the autonomous driving computer 110.

The autonomous driving computing system may be capable of communicatingwith various components of the vehicle. For example, returning to FIG. 1, computer 110 may be in communication with the vehicle's centralprocessor 160 and may send and receive information from the varioussystems of vehicle 101, for example the braking system 180, accelerationsystem 182, signaling system 184, and navigation system 186 in order tocontrol the movement, speed, etc. of vehicle 101. In one example, thevehicle's central processor 160 may perform all of the functions of acentral processor in a non-autonomous computer. In another example,processor 120 and 160 may comprise a single processing device ormultiple processing devices operating in parallel.

In addition, when engaged, computer 110 may control some or all of thesefunctions of vehicle 101 and thus be fully or partially autonomous. Itwill be understood that although various systems and computer 110 areshown within vehicle 101, these elements may be external to vehicle 101or physically separated by large distances.

The vehicle may also include a geographic position component 144 incommunication with computer 110 for determining the geographic locationof the device. For example, the position component may include a GPSreceiver to determine the device's latitude, longitude and/or altitudeposition. Other location systems such as laser-based localizationsystems, inertial-aided GPS, or camera-based localization may also beused to identify the location of the vehicle. The location of thevehicle may include an absolute geographical location, such as latitude,longitude, and altitude as well as relative location information, suchas location relative to other cars immediately around it, which canoften be determined with better accuracy than absolute geographicallocation.

The vehicle may also include other devices in communication withcomputer 110, such as an accelerometer, gyroscope or anotherdirection/speed detection device 146 to determine the direction andspeed of the vehicle or changes thereto. By way of example only,acceleration device 146 may determine its pitch, yaw or roll (or changesthereto) relative to the direction of gravity or a plane perpendicularthereto. The device may also track increases or decreases in speed andthe direction of such changes. The device's provision of location andorientation data as set forth herein may be provided automatically tothe user, computer 110, other computers and combinations of theforegoing.

The computer 110 may control the direction and speed of the vehicle bycontrolling various components. By way of example, if the vehicle isoperating in a completely autonomous driving mode, computer 110 maycause the vehicle to accelerate (e.g., by increasing fuel or otherenergy provided to the engine), decelerate (e.g., by decreasing the fuelsupplied to the engine or by applying brakes) and change direction(e.g., by turning the front two wheels).

The vehicle may also include components for detecting objects externalto the vehicle such as other vehicles, obstacles in the roadway, trafficsignals, signs, trees, etc. The detection system 154 may include lasers,sonar, radar, cameras or any other detection devices which record datawhich may be processed by computer 110. For example, if the vehicle is asmall passenger vehicle, the car may include a laser mounted on the roofor other convenient location.

As shown in FIG. 3 , vehicle 101 may include a small passenger vehiclehaving lasers 310 and 311, mounted on the front and top of the vehicle,respectively. Laser 310 may have a range of approximately 150 meters, athirty degree vertical field of view, and approximately a thirty degreehorizontal field of view. Laser 311 may have a range of approximately50-80 meters, a thirty degree vertical field of view, and a 360 degreehorizontal field of view. The lasers may provide the vehicle with rangeand intensity information which the computer may use to identify thelocation and distance of various objects. In one aspect, the lasers maymeasure the distance between the vehicle and the object surfaces facingthe vehicle by spinning on its axis and changing its pitch.

The vehicle may also include various radar detection units, such asthose used for adaptive cruise control systems. The radar detectionunits may be located on the front and back of the car as well as oneither side of the front bumper. As shown in the example of FIG. 3 ,vehicle 101 includes radar detection units 320-323 located on the side(only one side being shown), front and rear of the vehicle. Each ofthese radar detection units may have a range of approximately 200 metersfor an approximately 18 degree field of view as well as a range ofapproximately 60 meters for an approximately 56 degree field of view.

In another example, a variety of cameras may be mounted on the vehicle.The cameras may be mounted at predetermined distances so that theparallax from the images of 2 or more cameras may be used to compute thedistance to various objects. As shown in FIG. 3 , vehicle 101 mayinclude 2 cameras 330-331 mounted under a windshield 340 near the rearview mirror (not shown). Camera 330 may include a range of approximately200 meters and an approximately 30 degree horizontal field of view,while camera 331 may include a range of approximately 100 meters and anapproximately 60 degree horizontal field of view.

In addition to the sensors described above, the computer may also useinput from other sensors and features typical to non-autonomousvehicles. For example, these other sensors and features may include tirepressure sensors, engine temperature sensors, brake heat sensors, breakpad status sensors, tire tread sensors, fuel sensors, oil level andquality sensors, air quality sensors (for detecting temperature,humidity, or particulates in the air), door sensors, lights, wipers,etc. This information may be provided directly from these sensors andfeatures or via the vehicle's central processor 160.

Many of these sensors provide data that is processed by the computer inreal-time, that is, the sensors may continuously update their output toreflect the environment being sensed at or over a range of time, andcontinuously or as-demanded provide that updated output to the computerso that the computer can determine whether the vehicle's then-currentdirection or speed should be modified in response to the sensedenvironment.

In addition to processing data provided by the various sensors, thecomputer may rely on environmental data that was obtained at a previouspoint in time and is expected to persist regardless of the vehicle'spresence in the environment. For example, returning to FIG. 1 , data 134may include detailed map information 136, e.g., highly detailed mapsidentifying the shape and elevation of roadways, lane lines,intersections, crosswalks, speed limits, traffic signals, buildings,signs, real time traffic information, vegetation, or other such objectsand information. For example, the map information may include explicitspeed limit information associated with various roadway segments. Thespeed limit data may be entered manually or scanned from previouslytaken images of a speed limit sign using, for example, optical-characterrecognition.

FIG. 4 is an example of a highway 400. In this example, highway 400includes 3 northbound lanes 410-412 and 3 southbound lanes 420-22defined by broken lane lines 430-33 and solid lane lines 440-43. Highway400 also includes shoulders 450-51 defined between solid lane line 440and barrier 460 and solid lane line 441 and barrier 461, respectively.Between the northbound and southbound lanes, highway 400 includes amedian 470.

FIG. 5 is an example of map information 500 for highway 400 of FIG. 4 .Map information includes data indicating the location and orientation ofthe various features of highway 400. For example, map information 500includes northbound lane data 510-512 identifying northbound lanes410-412 as well as southbound lane data 520-522 identifying southboundlanes 420-22. Map information 500 also includes broken lane line data530-33 and solid lane lines 540-43 representing broken lane lines 430-33and solid lane lines 440-43. Shoulders 450-51 are also represented byshoulder data 550-551. Barriers 460-61 are represented by barrier data560-61, and median 470 is represented by median data 570.

The map information may also include three-dimensional terrain mapsincorporating one or more of objects listed above. For example, thevehicle may determine that another object, such as a vehicle, isexpected to turn based on real-time data (e.g., using its sensors todetermine the current GPS position of another vehicle and whether a turnsignal is blinking) and other data (e.g., comparing the GPS positionwith previously-stored lane-specific map data to determine whether theother vehicle is within a turn lane).

The map information 136 may also include autodrive zones which arecurrently available for autonomous driving. Autodrive zones may includefor examples, areas within the map information which have beenpre-approved or otherwise designated for initiating driving in anautonomous driving mode. These areas may include, for example, specificlanes on a highway, residential streets, etc. In this regard, autodrivezones may include pre-determined autodrive lanes. Areas which may beexcluded from autodrive zones may include, by way of example only,acceleration lanes, exit lanes, merges, intersections, toll booths,known construction zones, and school zones and portions of roadway nearsuch areas. Although computer 110 may restrict initiating the autonomousdriving mode in areas which are not designated as autodrive zones, thecomputer 110 may actually be fully capable of maneuvering the vehiclethrough such areas or actually initiating the autonomous driving mode.

For example, map information 600 of FIG. 600 includes map information500 and also autodrive zones 610 and 620. In this example, autodrivezone 610 includes the southbound lanes 430-32 (represented by southboundlane data 530-32) of highway 400 while autodrive zone 620 includes onlya portion of the northbound lanes 420-22 (represented by northbound lanedata 520-22) of highway 400. Autodrive zone 610 includes the zones oflanes 410-22 (represented by lanes 510-22); however, in this example,only lanes 410 (510) and 411 (511) include autodrive lanes 611 and 612,respectively. Similarly, autodrive zone 620 includes portions of thezones of lanes 420-22 (represented by lanes 520-22); however, in thisexample, only lanes 421 (521) and 422 (522) include autodrive lanes 621and 622, respectively. Thus, not all portions of highway 400 areautodrive zones, and not all lanes within autodrive zones are autodrivelanes.

Although the detailed map information 136 is depicted herein as animage-based map, the map information need not be entirely image based(for example, raster). For example, the map information may include oneor more roadgraphs or graph networks of information such as roads,lanes, intersections, and the connections between these features. Eachfeature may be stored as graph data and may be associated withinformation such as a geographic location whether or not it is linked toother related features. For example, a stop sign may be linked to a roadand an intersection. In some examples, the associated data may includegrid-based indices of a roadgraph to promote efficient lookup of certainroadgraph features.

Computer 110 may also receive or transfer information to and from othercomputers. For example, the map information stored by computer 110 (suchas the examples of map information 500 and 600 shown in FIGS. 5 and 6 )may be received or transferred from other computers and/or the sensordata collected from the sensors of vehicle 101 may be transferred toanother computer for processing as described herein. As shown in FIGS.7A and 7B, data from computer 110 within vehicle 101 may be transmittedvia a network to stationary computer 720 for further processing.Similarly, data from computer 720, such as software updates or weatherinformation as described below, may be transmitted via the network tocomputer 110. The network, and intervening nodes, may comprise variousconfigurations and protocols including the Internet, World Wide Web,intranets, virtual private networks, wide area networks, local networks,private networks using communication protocols proprietary to one ormore companies, Ethernet, WiFi and HTTP, and various combinations of theforegoing. Such communication may be facilitated by any device capableof transmitting data to and from other computers, such as modems andwireless interfaces. In another example, data may be transferred bystoring it on memory which may be accessed by or connected to computers110 and 720.

In one example, computer 720 may comprise a server having a plurality ofcomputers, e.g., a load balanced server farm, that exchange informationwith different nodes of a network for the purpose of receiving,processing and transmitting the data to and from computer 110. Theserver may be configured similarly to the computer 110, with a processor730, memory 740, instructions 750, and data 760.

As noted above, data 760 of server 720 may provide software updatesand/or weather related information. For example, server 720 may providesoftware updates to computer 110 including, for example, updating thevarious data stored in memory 130 such as instructions 132, detailed mapinformation 136, protocol data 138, and/or task data 140. In anotherexample, server 720 may receive, monitor, store, update, and transmitvarious information related to weather. This information may include,for example, precipitation, cloud, and/or temperature information in theform of reports, radar information, forecasts, etc.

Computer 110 may use protocol data 138 for assessing whether changing toor from autodrive mode will be permitted based on an assessment of thestatus of the vehicle's environment, the vehicle, as well as the driver.As described in more detail below, computer 110 may use protocols 138 toidentify preventive conditions and whether those preventive conditionsmay be corrected by the computer or the driver.

Task data 140 may include a series of tasks that can be performed by adriver controlling a vehicle such as vehicle 101. As discussed in moredetail below, these tasks may include closing a door, buckling aseatbelt, changing lanes, etc. Each task may be associated with acorresponding preventive condition such that the performance of the taskby a driver of a vehicle will correct a preventive condition identifiedusing protocol data 138. In this regard, task data may be stored as apart of protocol data 138 or separately.

The computer 110 may determine a proper order for each task. In thisregard, buckling a seatbelt may have a higher rank than changing lanes,as it would be unsafe for the driver to control the vehicle withoutwearing a seatbelt, etc. The task data may also include instructions anddisplay data for presenting tasks to the driver. For ease of reference,this disclosure will refer to the ranking data as being part of the taskdata 140, but the information required to determine the order of eachtask may be stored as part of protocol data 138, elsewhere in data 134and as part of the instructions 132.

In addition to the operations described above and illustrated in thefigures, various operations will now be described. It should beunderstood that the following operations do not have to be performed inthe precise order described below. Rather, various steps can be handledin a different order or simultaneously, and steps may also be added oromitted.

In one example, a driver of vehicle 101 may activate the autonomousdriving system 100 and use a user input 150 to request that the vehicleswitch from a manual driving mode to an autonomous driving or autodrivemode. Before granting the request, the computer 110 may determinewhether any conditions exist that will be used to prevent the switch.This may include accessing protocol data 138 and assessing the status ofthe vehicle's environment (current and likely future), the status of thevehicle and the vehicle systems, as well as the status of the driver toidentify any preventive conditions. For example, preventive conditionsmay include conditions indicating that it might be unsafe to switch frommanual to autonomous driving mode, that the vehicle is an area in whichautonomous driving is prevented by law, or that vehicle is not designedfor autonomous driving in the current environment (e.g., off-roadtravel).

The assessment of the status of the vehicle's environment may rely oninformation collected by the vehicle's various sensors, stored mapinformation, as well as information from a remote computer. For example,the assessments may include determining the status of the weather in thevehicle's immediate environment and whether this information indicatesthat it would be safer for the driver to retain full control of thevehicle. This may include receiving weather data from computer 720and/or detecting weather conditions from the vehicle's sensors.Generally, driver may be in a better position to navigate the vehiclethan the computer 110 when there is heavy or accumulating precipitationsuch as rain, snow, hail, etc. Thus, under such weather conditions,computer 110 may identify a preventive condition.

Other current environmental assessments may include examining thecurrent location and speed of the vehicle and determining whether thevehicle is at an acceptable location and speed for switching from manualto autonomous driving mode. The vehicle's location may be received ordetermined from data generated by geographic position component 144and/or acceleration device 142. This location may be combined with datafrom the various sensors of the detection system 154 as well as mapinformation 136. For example, computer 110 may assess whether thevehicle's actual location agrees with the vehicle's location relative tothe detailed map information, whether data received from the sensors ofthe detection system agrees with the corresponding detailed mapinformation, whether the vehicle is currently driving in an autodrivezone, whether the vehicle is currently driving in an autodrive lane,whether the roadway is paved (as opposed to dirt), whether the road iswide enough (as opposed to being too narrow for two vehicles to pass oneanother), whether the vehicle is on a straight roadway (as opposed todriving around a curve, down or up a hill, etc.), whether the vehicle issufficiently centered in the lane, whether the vehicle is surrounded byor boxed in by other vehicles (for example, in dense trafficconditions), whether the vehicle is currently in a school zone, whetherthe vehicle is facing oncoming traffic (for example, driving northboundin a southbound lane according to the map information 136), whetheranother object is pulling into the vehicle's lane proximate to thevehicle, whether the vehicle is at least some pre-determined minimumdistance from other vehicles or moving objects in the roadway, whetherthe vehicle must make any small maneuvers to avoid non-moving objects inthe roadway, whether the vehicle is traveling too fast or too slow, etc.If any of these conditions are not true, computer 110 may identify thatcondition as a problem condition.

In addition to the vehicle's current environment, computer 110 mayassess the driving environment that the vehicle is likely to encounterin the future. In this regard, rather than relying on data from thevehicle's detection system 154, computer 110 may receive and/or accessdata from a remote computer, stored map information, and user input.

In one example, the computer 110 may assess whether any upcomingsituations or maneuvers are likely to create a prevention condition. Forinstance, the computer may check the map data for upcoming preventionconditions if the driver were to maintain the vehicle's currenttrajectory or stay on a recommended route. The fact that a preventioncondition is likely to occur in the future may, itself, be considered aprevention condition that will prevent the user from activating theautodrive mode.

The amount of time that is expected to elapse until the futureprevention condition is likely to occur may be used to determine whetherthe user will be prevented from placing the vehicle into autodrive mode.In one example, there may be some minimum amount of time during whichthe vehicle can be driven autonomously before it reaches a locationwhere manual driving mode would be required, such as at a trafficintersection or merge. The lane of the road may continue, but thecurrent lane may cease to be an autodrive lane at the intersection. Thecomputer may thus estimate, based on the speed of vehicle or other data(such as the road's speed limit), the amount of time it will take forthe vehicle to reach the intersection. If the estimated duration is lessthan a threshold, the upcoming intersection may be considered a currentprevention condition. The threshold may be predetermined or dynamicallycalculated. For example, the threshold may be set to approximately 50seconds since users may consider it disruptive to put the car intoautodrive mode and then be forced out of autodrive mode within a span of50 seconds. The threshold may also be dynamically determined, i.e., thethreshold may be increased in bad weather.

Computer 110 may also use protocol data 138 to assess the status of thevehicle and the vehicle's various systems and determine whether there isa prevention condition. For example, computer 110 may determine whetherthe various features of the autonomous driving system 100 are workingproperly. This may include communicating with a remote computer, such ascomputer 720, to determine whether computer 110 has the most up to datesoftware. In addition, computer 110 may also communicate with thevarious components of autonomous driving system 100 to ensure that boththe computer 110 is able to communicate with these systems and also todetermine whether those systems are working properly. For example,computer 110 may determine whether the various sensors of detectionsystem 154 are working properly, responsive, properly calibrated, etc.In not, computer 110 may identify a problem condition.

In addition to communicating with the components of autonomous drivingsystem 100, computer 110 may also determine the status of the vehicle'svarious non-autonomous components. This may include querying thevehicle's central processor 160 for information regarding the status ofthe vehicle. In this regard, the vehicle's central processor 160 may bein communication with other vehicle features or sensors, such as thelights, drive system, tire pressure monitors, door sensors, etc.

Computer 110 may then use this information regarding the other vehiclefeatures and sensors to make various assessments. For example, computer110 may determine whether the vehicle has met minimum maintenancestandards (for example, oil changes are up to date), whether thevehicle's tires are properly inflated, whether the vehicle's doors areclosed, whether the vehicle is currently in “drive” (as opposed to“neutral,” “reverse,” or “park”), whether the vehicle is in two wheeldrive (for example, on a vehicle which allows for the driver to manuallyswitch into four wheel or all wheel drive), whether the vehicle'sautomatic wipers are currently on, whether the vehicle's headlights orfog lights are on, and whether no other warning indicators (such ascheck engine lights) are currently active. In not, computer 110 mayidentify a prevention condition.

Computer 110 may also use protocol data 138 to assess the status of thevehicle driver. For example, computer 110 may also query centralprocessor 160 to determine whether the driver's seatbelt is properlybuckled. Other examples may include using sensors or other methods todetermine driver sleepiness, driver intoxication, or whether the driveris authorized or not to use the vehicle.

The assessments need not be organized into distinct environmental,vehicle, or driver conditions but may actually comprise a largercollection or list of features for assessment. Alternatively, somefeatures identified as being associated with a particular statusassessment may be associated with other status assessments or checkedmultiple times. For example, whether the driver door is open may be anassessment made as part of a vehicle status assessment as well as adriver status assessment.

The assessments made by computer 110 according to protocol data 138 maybe performed in any order. In addition, many of these assessments may bemade simultaneously, for example by multiple processors operating inparallel. The assessments may be made repeatedly before receiving anautodriving request (or a request to use the autodrive mode), may bemade at the time of receiving an autodriving request, may be checkedrepeatedly after receiving an autodriving request, and any combinationof the foregoing.

Once computer 110 has identified a prevention condition that wouldprevent the transition from manual to autonomous driving mode, thecomputer may determine whether the computer or the driver can takeimmediate corrective action. If not, the computer 110 may generate anotification to the driver informing him that the autonomous drivingmode is currently unavailable. This notification may also includeinformation indicating the reason why the autonomous driving mode isunavailable. The notification may also be displayed to the driver, forexample, using electronic display 152.

For example, referring to the assessments regarding the status of thevehicle's environment, neither the computer 110 nor the driver couldchange current weather conditions. In this regard, computer 110 maygenerate and display a notification explaining that the autonomousdriving mode is currently unavailable due to the weather conditions.

In another example, referring to the status of the vehicle, neither thecomputer 110 nor the driver may be able to immediately change vehiclemaintenance issues, such as an oil change indicator, low tire pressureindicator, or other more serious issues with regard to the varioussensors of detection system 154. Similarly, neither computer 110 nor thedriver may be able to immediately change a prevention condition causedby the failure of sensor to provide data. For example, if the computer110 is unable to determine whether the driver is wearing his seatbelt,simply asking the driver may not be sufficient to ensure that theseatbelt is actually operational and that the driver is wearing it.Accordingly, neither computer 110 nor the driver may be able to correctsuch a problem condition, and again, computer 110 may generate anddisplay a notification explaining that the autonomous driving mode iscurrently unavailable due to the prevention condition.

If the computer 110 is able to take corrective action to address theprevention condition, the computer may do so. For example, if theassessment of the vehicle's status indicates a problem with thevehicle's sensors which can be corrected by computer calibration, thecomputer 110 may recalibrate the sensors and continue with any itemsremaining to be assessed. In another example, if computer 110 determinesthat it does not have the most up to date software, this preventioncondition may be corrected by getting a software update, for example,from computer 720. In another example, computer 110 may correct someprevention conditions with a short delay. Various other problemconditions may also be corrected by computer 110. In addition, thecomputer may also perform some or all of these corrective actionsproactively, rather than waiting for the driver to request a switch frommanual driving mode to autonomous driving mode. The computer correctiveactions may also be performed in parallel with one another and/or inparallel with driver performed corrective actions.

While computer 110 may be capable of correcting many different problemconditions, it may be safer to request the driver to address at leastsome prevention conditions rather than computer 110. In this regard, forat least some prevention conditions, computer 110 may use task data 140to generate tasks for such conditions.

For example, referring to the status of the vehicle's environment,computer 110 may generate tasks for changing the vehicle's locationand/or speed. Depending upon the prevention condition, these tasks mayinclude moving the vehicle to an autodrive lane, centering the vehiclein a lane, creating a sufficient space between the vehicle and someother moving object in the roadway, and speeding up or slowing down thevehicle. Similarly, referring to the status of the vehicle, computer 110may generate tasks for correcting prevention conditions with regard toan improperly closed or open door, using the gear shift to put thevehicle into “drive”, turning on automatic wipers, turning on lights,etc. Regarding the status of the driver, the computer 110 may generate atask requesting the driver to put on his seatbelt, etc.

As noted above, each task generated by the computer 110 using the taskdata 140 may be ranked. Thus, after generating all of the required tasksto correct any identified prevention conditions, the computer 110 mayuse the rankings to order the tasks and then present the ordered tasksto the driver. This may allow the computer 100 to provide the checklistto the driver in an order that promotes safety and is consistent withprerequisites. For example, a driver might be instructed to put on hisseatbelt before being asked to put the vehicle in “drive” and beforeasked to move to an autodrive lane.

Similarly, although the driver may be presented with multiple tasks atthe same time, it may be more helpful to provide each task to the driverone at a time, waiting until each task has been completed beforeproviding the driver with the next task. In addition, tasks whichrequire a longer time to complete may need to be followed by anotherdriver request to switch to the autonomous mode while other tasks whichmay be completed in a shorter time may not.

FIG. 8 depicts a series of screen images 810, 820, 830, and 840 whichmay be displayed to a driver, for example on electronic display 152.These screen images may identify tasks required to correct identifiedproblem conditions. In this example, computer 110 has already determinedthat each of these prevention conditions should and can be corrected bythe driver. The conditions include that the vehicle is not in “drive,”that the automatic wipers are not on, that the automatic lights are noton, and that the vehicle that the vehicle is not in an autodrive lane.Thus, computer 110 has generated a set of tasks and associated screenimages 810, 820, 830, and 840. As noted above, these tasks and screenimages are presented in a certain order.

In this example, the first task includes shifting the vehicle into“drive” as shown in screen image 810. Once the first task is completedby the driver, the computer 110 may determined that the correspondingcondition has been corrected. As a result, the computer 110 may show thenext task. In this example, the next task includes turning on automaticwipers as shown in screen image 820. Again, once the task of screenimage 820 is completed, the next task turning on automatic lights may bedisplayed as shown in screen image 830. Once this task is completed,computer 110 may display the next task, moving to an autodrive lane, asshown in screen image 840.

Accordingly, as described above, each screen image may be shown insequence, one at a time, until each task 810, 820, 830, and 840 has beencompleted and the computer determines that the corresponding preventioncondition no longer apply.

Once the identified problem conditions have been corrected by thedriver, the computer may allow the driver to switch from the manualdriving mode to the autonomous driving mode. For example, once all ofthe required tasks have been completed by the driver (such as thoseshown by screen images 810, 820, 830, and 840), computer 110 may displaya message, such as that shown in screen image 850, indicating that theautonomous driving system 100 and computer 110 is ready to take controlof the vehicle. In addition, computer 110 may change the color of all ora portion of the display to further indicate that the vehicle is readyfor autonomous driving mode. In this regard, screen image 850 mayinclude a different background color from screen images 810, 820, 830,and 840. Once ready, the computer may automatically switch from themanual driving mode to the autonomous driving mode or may wait for afurther request from the driver to ensure the driver is actually ready.

Before switching from manual to autonomous driving mode, computer 110may conduct the aforementioned assessments continuously. In this regard,even though the driver or computer 110 has corrected various impedimentsto placing the vehicle in autodrive mode, new problems may arise (or asnoted above, some problems may be corrected). For example, once thedriver has moved the vehicle 101 into an autodrive lane in response tothe task depicted by screen image 840 of FIG. 8 , the computer 110 mayidentify additional problems. In one example, these additionalpreventive conditions may include that the vehicle is not centered in alane, that the vehicle is then moving too fast, and that the vehicle istoo close to another object in the roadway.

Based on these additional conditions, computer 110 may generateadditional tasks including centering the vehicle in the lane, slowingthe vehicle down, and increasing the distance between the vehicle andthe another object in the roadway. As with the example of FIG. 8 ,computer 110 may also generate corresponding screen images to encouragethe driver to correct each of the conditions. As shown in FIG. 9 ,computer 110 may display to the driver, for example on electronicdisplay 152, a series of screen images 910, 920, and 930 identifyingtasks to be completed by the driver. As with the example of FIG. 8 ,each screen image may be displayed one at a time until the driver hascompleted the task and the computer determines that the correspondingcondition has been corrected. Again, once the computer 110 determinesthat there are no remaining or newly identified preventive conditions,computer 110 may display a message, such as that shown in screen image940, indicating that the autonomous driving system 100 and computer 110is ready to take control of the vehicle.

Flow diagram 100 of FIG. 10 is an example of some of the aspectsdescribed above which may be performed by a computer associated with avehicle having an autonomous driving mode and a manual driving mode. Forexample, computer 110 of vehicle 101 may receive an autodrive request ora request to switch from a manual driving mode to an autonomous drivingmode at block 1002. The computer 110 then responds by accessing protocoldata at block 1004. The protocol data is then used to assess the statusof the vehicle's current environment, the vehicle's future environment,the vehicle, and the driver at blocks 1006, 1008, 1010, and 1012,respectively. Again, these assessments may be conducted in any order.The computer 110 then determines if these assessments have identifiedany problem conditions at block 1014. If not, the computer 110 proceedswith the switch from the manual to the autonomous driving mode at block1016.

If as a result of the assessments, the computer 110 does identify one ormore preventive conditions, the computer determines whether the computeror a driver can correct those identified conditions at block 1018. Ifthe conditions cannot be corrected, the computer 110 displays a failuremessage to the driver at block 1020. As noted above, this failuremessage may indicate the particular issue for the failure.

If the computer 110 or a driver can sufficiently promptly correct theidentified problems, the computer may correct problems that can becorrected by the computer at block 1022. If at block 1024 no identifiedpreventive conditions remain, the computer 110 proceeds with the switchfrom the manual to the autonomous driving mode at block 1016.

Returning to block 1024, if after the computer corrects any of theproblems, other problems remain, the computer 110 generates acorresponding task or task using pre-ranked task data for each of theremaining problems at block 1026. The computer 100 then displays thehighest ranking task for completion by the driver of vehicle 101 atblock 1028. The computer continues to display the task until it iscompleted at block 1030. In this example, once the task is completed,the computer returns to block 1004 to access the protocol data andreassess the status of the various features of blocks 1006, 1008, 1010,and 1012. The process may continue until all of the identified problemsare corrected and the computer 110 proceeds with the switch from manualto autonomous driving mode at block 1016.

Alternatively, once a task is completed at block 1030, the computer maysimply display the next highest ranking task until all of the identifiedproblem conditions are corrected. Once this occurs, the computer 110proceeds with the switch from manual to autonomous driving mode.

Conducting the aforementioned assessments and performing correctiveactions can prevent certain problems, such as potential safety issues,actions that the user would consider disruptive, actions that may damagethe car over time, etc. For example, if computer 110 is unable to driveautonomously in a particular lane or a shoulder because of some knowncondition, it may be dangerous to begin driving autonomously in thatlocation. In another example, if the vehicle is too close to anothervehicle or driving too fast, switching directly into autonomous drivingmode may cause the computer to slow the vehicle down immediately anddramatically. This may be uncomfortable and also disconcerting for thedriver. In yet another example, if the vehicle is not sufficientlycentered in a lane, computer 110 may surprise the driver by moving intoa lane in which the driver did not intend to drive. Abrupt lane changesmay make the driver nervous or uncomfortable. Thus, the featuresdescribed herein allow the driver to maintain control until both thecomputer 110, and the driver, are assured that it is safe to switchdriving modes.

Once the vehicle is in the autonomous driving mode, if any conditionsarise which require a switch to manual driving mode, computer 110 maycontinue at least some of the assessments as described above and alsomade other assessments regarding the continued safety of driving inautonomous driving mode. If any preventive conditions are identifiedwhich cannot or should not be corrected immediately by computer 110, thecomputer 110 may inform the driver of the need to switch back into themanual driving mode. For example, if the computer 110 detects someemergency or some preventive condition such as a loss of communicationwith some component of the autonomous driving system, the computer 110may immediately inform the driver of the need to switch to the manualdriving mode. By doing so quickly, even when the preventive conditiondoes not require immediate attention, the computer may provide thedriver with as much time as possible to take control of the vehicle.

Under certain specific conditions, the computer 110 may also discouragethe driver from switching from autonomous mode and into the manualdriving mode. Each vehicle may include an emergency manual override thatallows the driver to switch to the manual driving mode under anyconditions, but under normal operation there may be instances whereswitching to manual mode is prevented or delayed.

For example, after the first few seconds in autonomous driving mode, thecomputer 110 may detect that the driver is attempting to disengage theautonomous driving mode, such as by turning the steering wheel orhitting the accelerator. Because the request to disengage occurs soquickly after the request to engage, the computer 110 may determine thatthe request to disengage was unintentional. In this regard, rather thanautomatically disengaging, computer 110 may provide a warning to thedriver that the autonomous driving mode is disengaging and also continueto control some aspects of the vehicle until further actions by thedriver, such as by continuing to turn the steering wheel or hit theaccelerator, which indicate that the driver was indeed attempting todisengage the autonomous driving mode.

In another example, the computer 110 may prevent the driver fromswitching to manual driving mode when the vehicle is performing anaction that cannot be easily transferred to the driver before the actionis complete. For example, it may not be easy to safely return control tothe driver in the middle of a sharp turn. In this regard, computer 110may continuously make this assessment and provide a warning during thosetimes when the computer determines that switching to manual driving modewould be unsafe or uncomfortable for the driver. This warning may bechanged or removed during times when the computer determines thatswitching to manual driving mode would be safe and comfortable.

As these and other variations and combinations of the features discussedabove can be utilized without departing from the subject matter asdefined by the claims, the foregoing description of exemplaryembodiments should be taken by way of illustration rather than by way oflimitation of the subject matter as defined by the claims. It will alsobe understood that the provision of the examples described herein (aswell as clauses phrased as “such as,” “e.g.”, “including” and the like)should not be interpreted as limiting the claimed subject matter to thespecific examples; rather, the examples are intended to illustrate onlysome of many possible aspects.

The invention claimed is:
 1. A method comprising: receiving, by one ormore processors from an occupant of a vehicle, a request to operate thevehicle in an autonomous driving mode; assessing, by the one or moreprocessors, whether a seatbelt of the occupant is buckled; generating,by the one or more processors, a task request for the occupant to bucklethe seatbelt based on at least the assessment; displaying, by the one ormore processors on a display of the vehicle, the task request to theoccupant of the vehicle; and only upon determining that the task requesthas been completed, allowing, by the one or more processors, the vehicleto operate in the autonomous driving mode in response to the request. 2.The method of claim 1, wherein displaying the task request is inresponse to at least one the assessment.
 3. The method of claim 1,further comprising, continuously updating the assessment.
 4. The methodof claim 1, further comprising: determining that the task request iscompleted; and displaying the task request until the task request isdetermined to be completed.
 5. The method of claim 4, furthercomprising, once the task request is completed, displaying a second taskrequest for a problem condition.
 6. The method of claim 4, furthercomprising, after determining that the task request is completed,reassessing whether a seatbelt of the occupant is buckled.
 7. The methodof claim 6, further comprising, while controlling the vehicleautonomously, displaying a second task request for the occupant.
 8. Themethod of claim 7, wherein the second task request requests that theoccupant buckle the seatbelt.
 9. The method of claim 1, wherein arequest of the assessment indicates that the one or more processors areunable to determine whether the seatbelt of the occupant is buckled,displaying a notification that an autonomous driving mode isunavailable.
 10. The method of claim 1, further comprising, determininga priority for the task request, and wherein displaying the task requestis further based on the priority.
 11. A system comprising one or moreprocessors configured to: receive from an occupant of a vehicle, arequest to operate the vehicle in an autonomous driving mode; assesswhether a seatbelt of the occupant is buckled; generate a task requestfor the occupant to buckle the seatbelt based on at least theassessment; display on a display of the vehicle, the task request to theoccupant of the vehicle; and only upon determining that the task requesthas been completed, allow the vehicle to operate in the autonomousdriving mode in response to the request.
 12. The system of claim 11,wherein the one or more processors are further configured to display thetask request is in response to at least one the assessment.
 13. Thesystem of claim 11, wherein the one of more processors are furtherconfigured to continuously update the assessment.
 14. The system ofclaim 11, wherein the one or more processors are further configured to:determine that the task request is completed; and display the taskrequest until the task request is determined to be completed.
 15. Thesystem of claim 14, wherein the one or more processors are furtherconfigured to, once the task request is completed, display a second taskrequest for a problem condition.
 16. The system of claim 14, wherein theone or more processors are further configured to, after determining thatthe task request is completed, reassess whether a seatbelt of theoccupant is buckled.
 17. The system of claim 16, wherein the one of moreprocessors are further configured to, while controlling the vehicleautonomously, display a second task request for the occupant.
 18. Thesystem of claim 17, wherein the second task request requests that theoccupant buckle the seatbelt.
 19. The system of claim 11, wherein arequest of the assessment indicates that the one or more processors areunable to determine whether the seatbelt of the occupant is buckled,wherein the one or more processors are further configured to display anotification that an autonomous driving mode is unavailable.
 20. Thesystem of claim 11, wherein the one or more processors are furtherconfigured to determine a priority for the task request, and whereindisplaying the task request is further based on the priority.